Designing Comprehensible Agents

For many applications, it is important that agents be not only correct, but also comprehensible to human users. Typically, people have tr ied to make agents' behavior and reasoning understandable by adding post-hoc specialpurpose explanation systems, wi th often disappointing results. Here, I instead take the comprehensibility of agent behavior as a cent ra l agent design consideration from the start. I describe an agent architecture, the Expressivator, that supports comprehensibility on top of a behavior-based framework, using four technical innovations: (1) structuring the agent's behavior according to the signs and signifiers it is intended to communicate; (2) allowing the agent to keep track of its impression on the user wi th sign management, (3) using behavioral transitions to explain the reasons for agent, behavior, and (4) expressing behavioral interrelationships directly using meta-level controls.